Movement is an important aid to human visual perception. It helps us see, interpret and better understand our surroundings. Despite its usefulness to human viewers, motion is often the bane of photography. The clearest, most detailed image requires a perfectly stationary camera and scene. This is extremely difficult for amateur photography in natural settings.
Conventional cameras include several manual or automatic controls to deal with various camera and scene parameters, such as focus depth and exposure time. However, the solutions for dealing with motion in a scene are limited. Typically, the exposure time is decreased as the amount of motion increases.
Current imaging practice generally follows an ‘instantaneous’ ideal, a computation-free, zero-order model of motion selection. Ideally, the exposure time is made the longest that is possible so that moving objects still appear substantially motionless.
It is desired to provide improved sensing methods that will enable digital cameras to use a first-order motion model.
Motion blur is the result of relative motion between the camera and the scene during integration or ‘exposure time’ while acquiring an image. Motion blurred images can be restored up to lost spatial frequencies by image deconvolution, provided that the motion is shift-invariant, at least locally, and that a blur function, also known as a point spread function (PSF), that caused the blur is known.
However, image deconvolution belongs to a class of ill-posed inverse problems for which the uniqueness of a solution cannot be established and the solutions are oversensitive to perturbations in the input data. Several techniques are known for motion deblurring and reblurring.
Exposure Time Solutions
Shortening the exposure time is a common solution. However, a short exposure time increases noise and unnecessarily penalizes static areas of the image. A high speed camera can capture fast motion, but that is expensive in terms of sensing, bandwidth and storage. A high speed camera also fails to exploit inter-frame coherence. Often, high speed cameras require bright lights. Visually stunning results for high speed objects can be obtained by using a modest exposure time but an extremely narrow-duration flash. However, strobed flash is often impractical in outdoor or distant scenes. In addition, flash only captures an instant of the action and fails to indicate the general movement in the scene.
Smarter Cameras
To overcome camera motion, adaptive optical components can be physically stabilized using inertial sensors that compensate for camera motion. Alternatively, some CMOS cameras perform high-speed frame captures within normal exposure time, enabling multiple image-based motion blur removal. Those techniques are able to produce clear and crisp images, given a reasonable exposure time.
A hybrid imaging system can estimate the PSF using an auxiliary low resolution high frame rate sensor. An accurate PSF makes deblurring possible, even with a long exposure. Those methods compensate for camera motion but do not respond to object motion within the scene.
Video Analysis
Partial information can be combined to estimate and deblur videos based on successive frames of captured by a video camera, or from frames captured by multiple co-located cameras with overlapped exposure times.
Post-Processing Solutions
There are two main classes of methods for deblurring an image given the blur PSF. The first class, in the frequency domain, can use a Wiener filter or a regularized inversion, and entails computation of a Fourier (cosine) transform of the deblurred image. The second class includes iterative updates approaches. These include the Lucy-Richardson algorithm and other updates that iteratively optimize loss indices based on image statistics.
A noisy, short exposure image can also be improved by using color constraints observed in a long exposure photo. Blind deconvolution is widely adopted to enhance a single blurred image, based on various assumptions applied to the PSF. PSF estimation remains a challenging problem for arbitrary motions. And even when the PSF is known, deblurred images are often significantly inferior to the original image due to amplified noise, resampling and quantization issues. Often, it is only possible to deblur small movements.
Coded Sampling
Binary and continuous codes are commonly used in signal processing to modulate signals with a broadband response. The codes include ‘chirps’ that sweep the carrier over a wide frequency band during the pulse interval. Maximum length sequences (m-sequences) and modified uniform redundant arrays (MURA) are popular choices for coding and decoding by circular convolution. Coded-aperture astronomical imaging uses MURA codes to improve the signal to noise ratio while capturing X-ray and gamma-ray wavelengths unsuitable for conventional lenses.
Broadband signals have applications in a range of technologies, such as spread spectrum coding for noise-robust communication and code division multiplexing (CDMA) to minimize interference with other channels. Acousticians use m-sequences to design two dimensional panels that exhibit minimal sound diffraction.
Consider the problem of deblurring a 1-D signal via deconvolution. The goal is to estimate a signal S(x) that was blurred by a point spread function P(x) of a linear system. Then, a measured image signal I(x) is known to beI(x)=P(x)*S(x),  (1)where * denotes convolution. In the ideal case, a good estimate of the image, S′(x), can be recovered via a deconvolution filter P+(x), such thatS′(x)=P+(x)*I(x).  (2)
In the case of band-limited point-spread functions or point spread functions with incomplete coverage of the Fourier domain, information is lost and deconvolution is not possible. For example, capturing an image with exposure duration T is equivalent to a convolution with a box filter in the temporal domain. The resultant alteration is a flat blur. In the frequency domain, the signal is multiplied by a band-limited synchronization function with zeros at the intervals of 2/T and significant attenuation at most other frequencies.
To overcome this problem, several methods select their reconstruction from the range of possible solutions using an iterative maximum likelihood estimation approach. One well-known class of techniques use a statistical model for image formation based on a Bayes formula. The Richardson-Lucy algorithm is a non-linear ratio-based method that produces non-negative gray level values. The iterative deconvolution technique is applicable for whole motion blur and assumes the complete signal I(x) is available. But iterative deconvolution fails to handle cases where parts of the scene have different PSFs, such as in the case of a moving object in front of a static textured background. When a part of the moving object is occluded, some values of I(x) are unobservable.